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ABSTRACT 



The methodology of using a mixed qualitative and 
quantitative research design to evaluate 16 " inclusive” preschool programs in 
a continuing study is detailed. The qualitative approach was used to 
understand the general phenomenon and integrated with more quantitative, 
structured, and precise measures in an iterative, sequential process to 
develop repeated data collection activities and expand data interpretation. 

At each site, five children with disabilities, two typically developing 
peers, the children's families, direct service providers, and administrators 
and policy makers were involved in the study. The qualitative phase involved 
participant observation, semi-structured interviews, and document analysis. 
Quantitative measures encompassed survey questionnaires and measures of child 
behavior, friendship patterns, and the educational context. The process of 
data reduction, comparison, and integration for both qualitative and 
quantitative phases is detailed. Systematic comparison of information from 
interviews and survey data is illustrated in a table. Comparison involved the 
following themes: (1) how and why the child was placed in the program; (2) 

the program's appropriateness for the child; (3) the meaning of inclusion; 

(4) peer relationships; (5) helpful and non-helpful players; and (6) the 
child's participation in family and community activities. (DB) 
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Making the Whole More Than the Sum of the Parts: 

Challenges in a Mixed Method Study of Inclusion (aera97#2.doc; 2/27/97) 

Recent practice has seen increased use of mixed method research designs that 
capitalize on the strengths of qualitative and quantitative methods. While considerable 
theoretical development about the purposes and analysis of mixed method designs has 
occurred (Caracelli & Greene, 1993; Cook & Reichardt, 1979; Greene, Caracelli, & 
Graham, 1989; Shotland & Mark, 1987; Smith & Louis, 1982), many conceptual, 
technical, and practical issues remain in the implementation of these designs. The 
purpose of this paper is to describe methodological issues in the collection and analysis 
of extensive quantitative and qualitative data in a national cross-site study of preschool 
inclusion. 

Although much has been written about theoretical and paradigmatic issues in 
mixed method designs, there is a gap in the literature about practical issues in analyzing 
and interpreting data in these designs. This paper attempts to fill that gap in that it 
focuses on those practical aspects in analysis and interpretation from a complex and rich 
data set. We discuss how multiple methods were combined in separate classroom and 
family studies to illustrate the principles of triangulation, development, complementarity, 
and expansion, as defined in the framework for mixed method designs proposed by 
Greene et al. (1989). Further, the paper will discuss issues confronted in integrating data 
from the different data sources and offer practical suggestions for analysis strategies. 

Overview of Research Institute 

The Early Childhood Research Institute on Inclusion (ECRII) is funded to 
conduct a five-year research program on issues related to the inclusion of preschool-aged 
children (ages 3 through 5) with disabilities in education and child care settings with 
typically developing peers. The cluster of research studies conducted by the institute are 
designed to examine barriers to inclusion as well as to identify and design strategies for 
overcoming those barriers and supporting inclusion. The research addresses five content 
areas: (a) ecological analysis of inclusion, (b) classroom intervention and professional 
collaboration, (c) family perspectives and community inclusion, (d) social policy, and (e) 
action research and knowledge utilization. 

This five-year institute is a multi-site effort of research investigators from five 
universities (San Francisco State University, Vanderbilt University, University of 
Maryland, University of North Carolina, University of Washington). This national, cross- 
site approach assumes that inclusion can be understood only by examining the many 
contexts in which children and their famili es participate. Thus, the ecological systems 
model articulated by Bronfenbrenner (1979) provides the guiding framework for this 
inquiry. Based on this conceptual model, the investigators have studied preschoolers 
within their classrooms, homes and community programs (microsystem level), the 
relationships between families and program professionals and among professionals across 
programs and settings (mesosystem level), the program administrative structures and 
policies and contexts for inclusion (exosystem level), and the larger socio- 
political/cultural contexts for inclusion (macrosystem level). 
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Method 

In order to understand the definitional constructs and contexts of inclusion both 
qualitative and quantitative measures were employed in this investigation- The 
investigation was structured such that the qualitative approach was used to understand 
the general phenomenon and from that understanding more quantitative, structured, and 
precise measures of aspects of child behavior and educational context were derived. 
These quantitative measures, then in turn, informed the investigators of areas where 
additional in-depth conceptual understanding was needed using a more qualitative 
approach. 

For a common conceptual framework, we believe it is useful to review here the 
five mixed methods purposes proposed by Greene et al. (1989): 

(1) Triangulation: seeks convergence, corroboration, or correspondence of results from 
the different methods; 

(2) Complementarity: seeks elaboration, illustration, or clarification of results from one 
method with the results of the other; 

(qualitative and quantitative methods are used to measure overlapping but also different 
facets of a phenomenon, yielding an enriched, elaborated understanding of that 
phenomenon.) 

(3) Development: uses the results from one method to help develop or inform the other 
method; 

(4) Initiation: seeks the discovery of paradox and contradictions, new perspectives or 
frameworks, or the recasting of questions or results from one method with questions or 
results from the other; and 

(5) Expansion: extends the breadth and range of inquiry by using different methods for 
different inquiry components. 

We will provide examples of how multiple methods were used for these different 
purposes in classroom and family studies described later in the paper. 

Sampling 

Program selection. Data for the ecological systems study were collected from four 
university sites located around the United States. At each site, investigators selected 4 
preschool programs within their area that were nominated by key informants as programs 
that were "inclusive." Thus, a total of 16 preschool programs were involved in this in- 
depth study. Purposive sampling was employed in order that the sample represented the 
range of inclusive options and models, and the range and diversity of children and 
f amili es in this country. Programs were operated by a number of different organizational 
systems including public schools, Head Start, and child care organizations, and they 
represented a range of service delivery models. 

Subject selection. Within each preschool program, 7 children were selected 
purposively for this study; five children were disabled and had an Individualized 
Education Program (IEP) and 2 were typically developing peers who attended the same 
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classes. In addition, the children’s families, direct service providers (e.g., teachers, 
paraprofessionals, related service providers), and the administrators and policy makers 
for these programs were studied. A total of 112 children and their families across these 
sites were included in this investigation. 

Measures 

The primary strategies employed in the qualitative approach were participant 
observation, semi-structured interviews, and document analysis. These strategies provided 
an insider or emic perspective in that the perspectives of the individuals involved in the 
inclusion process were examined through first-hand experience. Investigators served as 
participant observers in children’s classrooms examining children’s participation with 
peers and service providers, contacts between family members and professionals, and the 
relationships among and between participants at all levels of the system. Post-CASPER 
notes were taken following the CASPER structured observation sessions (see the next 
paragraph) to supplement the quantitative data obtained. A friendship survey regarding 
children’s friendship patterns at school and in the community was completed by both 
teachers and parents. Semi-structured interviews also were conducted with family 
members, direct service providers, administrators, and policy makers. In addition, key 
documents were collected and analyzed. 

Quantitative measures encompassed a number of measures of child behavior, 
friendship patterns, and contextual variables. More specifically these measures included 
the CASPER (Code for Active Student Participation and Engagement-Revised), a direct 
observation in the classroom of group arrangement, group composition, type of activity, 
activity initiation, child behavior, child’s social behavior, and teacher behavior. Further, a 
sociometric assessment, the Peer Rating Assessment, was administered to children. 

The Battelle Developmental Inventory was administered to all children to establish 
children’s developmental level. In addition, a survey of classroom activities that gathered 
information regarding modifications for children’s special needs was conducted. Finally, a 
questionnaire survey measure of family perception of inclusion experiences was 
developed and data were collected through a semi-structured telephone interview. 

The qualitative and quantitative measures used are separated into classroom and 
family domains as shown in Table 1. The rest of this paper focuses on information from 
two different types of ECRU studies: one study measured the child’s participation and 
social relationships in the classroom, while the other study focused on parents’ 
perceptions of the child’s inclusive program, family supports, as well as the child’s social 
relationships and participation in activities outside the classroom. For each study, we 
present information on specific measures used and discuss issues in analyzing and 
interpreting the qualitative and quantitative data collected. 
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Table 1. Classroom and Family Measures in the Ecological Systems Study 



Qualitative Measures 


Quantitative Measures 


Classroom: 

--Participant observation* 
-Post-CASPER notes* 

-Teacher friendship survey* 
—Interviews with teachers and other 
service providers* 

—Document analysis 


Classroom: 

-CASPER (Code for Active Student 
Participation and Engagement-Revised)* 
-Peer Rating Sociometric Assessment* 
-Battelle Developmental Inventory 
—Curriculum Activities Survey 


Family: 

-Face-to-face interviews* 
-Family friendship survey 


Family: 

-Telephone survey* 



Note: Only the asterisked measures are discussed in this rest of this paper. 

Classroom Studies Example 

In this section we describe the analytic procedures we used for the quantitative 
measures of child participation (CASPER) and social relationships (peer rating), and the 
qualitative data collected as field notes from participant observation in the classroom, 
post-CASPER notes, the teacher friendship survey, and from teacher interviews. 

Data Analysis 

The general framework for analysis and integration of the quantitative and 
qualitative classroom data is presented in Figure 1. For each data type, we proceeded 
through these steps: (1) data reduction through graphing and coding, (2) data 
transformation, (3) comparison of summarized data of both types, and (4) integration of 
the data into a case study report. Each step is described in more detail below. The data 
presented for illustration are based on a small, purposive sample of 7 children from one 
inclusive program. 



[Insert Figure 1 here] 



Data Reduction 

To manage this large body of data, the first step we took was to go through a data 
reduction process in which the quantitative data are displayed in tables and graphs and 
the qualitative data summarized. Our quantitative data consists of two sets. One is 6 half- 
hour time sampling of CASPER data for each of the 7 children in the program. The 



Figure 1 . Analytic Framework for Classroom Data 
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other is a peer rating scale that ranks the popularity of each child in relation to others in 
the same class. For the CASPER data, we initially ran a quick frequency tabulation by 
each of the seven categories (e.g., group arrangement, activity initiation, child’s social 
behavior, teacher behavior, etc.) and produced a series of graphs displaying the 
corresponding data. We also collapsed the data for typically developing children and 
children with disabilities in order to investigate the possible differences between the two 
groups. 

For the peer rating scale data, we generated two types of graphs. The first is a 
frequency distribution of the average scores for all the children in the class. The other is 
a display of all the scores a child received. Graphing the quantitative data allows us to 
examine the data visually and detect any apparent patterns and irregularities in the 
initial stage of data analysis. 

We went through a similar data reduction process for the qualitative data. For the 
Post-CASPER observation notes, we used a pre-designed form to highlight the key 
findings from 6 CASPER observations for each child on a single sheet. Findings from the 
teacher’s friendship survey were reduced to a paragraph length summary. 

Observation fieldnotes and teacher interview transcripts were entered into the 
Ethnograph 4.0, a computer program designed for the analysis of textual data. Each 
investigator then read through the computer printouts of fieldnotes and interviews and 
came up with emerging themes. The data were then coded according to a set of thematic 
categories agreed upon by the site team of researchers. Some of these categories, such as 
adult support and child social behavior, actually reflect the same categories in the 
quantitative measure of CASPER. In other words, the two methods shared some of the 
investigative focus. 

After all the text data were coded, the computer program was used to search 
through the entire database for the segments coded under the same thematic category, 
sort them out by child and print out the results. On the basis of these printouts, we wrote 
child case summaries regarding his or her participation, peer relationships, adult support, 
and so on. 

Data Transformation 



To pave the way for data comparison and integration, data reduction was 
performed at a higher level when some of the qualitative data were transformed into a 
quantitative format and vice versa. 

The post-CASPER observation notes were originally designed to be part of our 
qualitative data set. They contained specific examples of what the observer saw about a 
child’s positive and negative behaviors and the child’s role in adult and peer interactions, 
During the first phase of data reduction, the post-CASPER form had already been 
reduced from six sheets to one sheet. During the data transformation stage, we reduced 
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the summary further into some quantifiable data. For instance, we made a graph 
indicating the percentage of positive social interactions among a child’s total recorded 
positive behaviors. We also computed the percentage a child was marked to "blend in," 
"stick out," or "stand out." These numerical versions of the qualitative data made it easier 
to make visual comparisons between the qualitative and quantitative data sets. 

Another data transformation strategy for qualitative data was to compute the total 
number of lines a certain coded segment takes. For instance, in one program, we found 
that the teachers were very fond of one child with disabilities. They talked about her 
without our prompting them. The very fact that they spent a lot of time talking about 
this child suggested to us the special role she played. 

Data transformation for quantitative data took the form of written narratives that 
summarized the most salient points in tables or graphs. For instance, when the CASPER 
data was graphed, only the part that seemed to tell a child apart from the others was put 
into narratives. 



Data Comparison 

The next step in our data analysis was to compare the data from the two sources. 
By now, our data sets have taken on a new form as the result of the data reduction 
process. We have two sets of graphs, one from the quantitative data, and the other from 
the qualitative data. The same thing is true for the two sets of summaries, one was based 
on coded observation notes and teacher interviews, and the other represented the most 
salient points from reading the graphs. Data comparison was done between the two sets 
of graphs and summaries for the purposes of triangulation, complementarity and 
initiation. 



Triangulation . One of the benefits of using a mixed method design is that research 
findings get triangulated across multiple data types and sources. As we compared the 
graphs and summaries, we found a lot of convergence and corroboration in our data set. 
For instance, a precursor to the understanding of barriers to inclusion is to develop a 
detailed profile for each participating child. Ideally this profile would include a portrait 
of what a typical day was like for a child, how s/he interacted with peers and adults, and 
when s/he encountered difficulties. As we worked on this kind of child profile, we tried 
to make sure that the portrait that we painted for each child was accurate and 
me aning ful. We did this by checking whether the data from two different sources and 
types gave the same description. 

For instance, Mary is a girl with mental retardation. But that did not prevent her 
from becoming the most sociable child in an inclusive program of 30 children. Her 
gregarious nature was evident in both of the following two summary excerpts. 



The CASPER results indicate that Mary is the most sociable and active child among the seven 
children observed. She interacted positively with adults for 9.2% of the time (the highest number of 
all 7 children), while the average rate for her disabled peers was only 3.5%. Her interaction with 
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peers was at 15 %, which tied her with one of the typically developing peers. And she had the lowest 
percentage rate of 81% for "having no social behavior." In fact, five out of six of her "stand out" 
(positive) behaviors occurred during her social interaction. For example, she offered toys to peers, 
showed peers how to paste a rectangle on paper, played a computer game with a friend and 
constantly talked to adults and peers during free play. She was the initiator of an activity for a 
remarkable 25.8% of the time, which was much higher than the average 14.6% maintained by 
children with disabilities in her class as a whole. 

Three girls are sitting in front of the furniture from the house area saying that they’re going to 
church. Again, Mary includes herself in the group by pulling over a rocking chair and saying, "I going 
church"... She sits there momentarily but then joins Lani’s (the special education teacher] group 
again. After a few minutes, Mary discovers that Tonya has taken over her chair. She tells her to get 
up. Tonya does, but after Mary sits down, she makes room on her chair for Tonya who sits back 
down on the corner of the chair. In a few minutes, Mary brings over another chair for Tonya. Mary 
starts bringing over all the chairs from the house area and putting them in a line. She directs one kid 
after another to "sit down right here." 

The first excerpt is based purely on quantitative data collected by the CASPER 
over a period of 3 weeks by 2 observers. The second excerpt was a summary of fieldnotes 
of participant observation taken by a third investigator. The date and time when these 
data were collected did not overlap, but both captured the same characteristic of the 
target child. 

In other instances, however, triangulation was difficult to achieve due to the 
differences between the two methods. One such discrepancy that affected efforts to 
establish direct links between the qualitative and quantitative data had to do with the a 
priori nature of the ecological categories of the CASPER compared to the more 
"grounded" nature of the categories that emerged from the qualitative description. In the 
unstructured observations at one of the sites, it appeared that the amount the adults 
structured the task (i.e., gave instructions for an art project, directed a group play activity 
that children could later do by themselves, etc.) an/or participated rather than observed, 
appeared to have important effects on child involvement. Because these emergent 
categories did not have a one-to-one correspondence to the a priori ecological categories 
of the CASPER, it was not possible to use the quantitative data to achieve triangulation 
on this question. 



Complementarity . Using mixed methods enables one data source and type to 
complement the other. Qualitative and quantitative methods have different descriptive 
strengths (Firestone, 1987). While numbers can assess empirical reality more precisely, 
they are incapable of providing details on the process that lead to a certain kind of 
situation. Qualitative data, on the other hand, are often rich in process-oriented 
information. So combining the two can help ensure a more complete understanding of 
the topic under study. 



For example, when we looked at one of the graphs of the CASPER data, we 
noticed that one of the children with disabilities really stood out. In comparison with her 
peers, she spent the longest time staying by herself outside a group setting and the least 



time in circle time, which happened to be the main instructional tii 
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these numbers tell us something about the participation pattern for this child, we were 
more interested in finding out why she stayed that way. So we examined the graphs made 
from Post-CASPER observation notes, the qualitative portion of our data set. We found 
that all of her six negative behaviors had something to do with her difficulties in staying 
with a group. This suggested that her problem was not confined to circle time alone, but 
might have occurred in any of the teacher-directed group activities such as work group, 
story reading time and even gym class. 

An examination of the fieldnotes and teacher interviews further revealed that the 
child was actually given tacit permission by the special education teacher to leave the 
group whenever she felt like it. This liberal policy actually reflected the teacher’s 
educational philosophy. She believed that all children should try their best, but if they 
still could not follow the rules, they should be given time to learn. It is no use enforcing 
the rules when the children were not ready. The special education teacher said, "I don’t 
judge a child being a model student if they sit with their mouth closed, their legs are 
folded and they follow directions. I don’t agree with that." 

While one data type was successfully used on several occasions to elucidate 
something of the nature of the other data type, there were other instances where the 
ease of interpretation of one data type was reduced when seen in the light of the other. 
One set of research questions involved a straightforward comparison of the CASPER 
data between programs and between groups (children with and without disabilities). 

One of the advantages of this data that led to the inclusion of this instrument was that it 
would yield precise measures of program variables that would be objective and 
comparable across sites. The possession of qualitative data from the same enviro nm ents, 
however, gave rise to some doubts about the meanings of CASPER findings that might 
not have existed had a purely quantitative study been conducted. 

For example, one of the Post-Casper observations noted that during the CASPER 
observation session the teacher had a nondisabled peer accompany a child with a 
disability to hang up his coat. The children began discussing the child’s picture in his 
cubby. The teacher came over and helped them elaborate the discussion by going to 
each of the cubbies and naming and describing the pictures of all the children. As they 
became engaged in this activity the teacher left the center, but continued to monitor it 
from a different location. The two children eventually moved from the cubbies to the 
book center where they continued the same kind of interaction around the pictures in 
the books there. The CASPER totals for the observation would lead one to believe that 
more peer interaction occurred when children were alone and the critical role of the 
teacher would be lost. There were other cases where the CASPER codes were the same 
but the function of the episode was very different. In one classroom, field notes showed 
that toileting, handwashing and toothbrushing were conducted during ongoing free play 
activities and all children participated in turns. In another program, the children with 
disabilities conducted these activities after the other children had finished cleaning up 
and left for recess. These events would appear to be the same in the cumulative 
CASPER data despite the different effects it may have had on the opportunities for 
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social interaction and perceptions of the children involved. 

It was possible to match these data on a case by case basis in such a way as to 
achieve complementarity for our case studies, but when the CASPER data is merged 
across programs, knowledge obtained through qualitative means makes interpretation of 
the quantitative data less, rather than more, straightforward. With regard to the two 
examples presented here, we must ask what any ultimate differences in adult 
participation or time spent in self help activities really means, in a way we would not 
have had any knowledge of the sequential and contextual factors afforded us by our 
collection of qualitative information. 

Initiation . As expected, our two data types did not always corroborate or 
complement each other. Instead, they sometime seem to contradict each other. 

Whenever this happened, we tried to go beyond the data itself by looking at its sources, 
the context under which the data were collected, and the methods by which they are 
collected. 

In our interview with teachers in one program, three out of four teachers spoke 
highly of Mary, the previously-mentioned child with special needs. Sociable and sensitive, 
this child played and made friends with everybody in her class. The teachers especially 
loved the way Mary went around hugging her peers each day before she left for home. 
They considered Mary a role model to other children for showing love and being nice to 
friends. They thought Mary was the most popular child in the class. The special 
education teacher even wrote on the friendship survey that Mary’s disability did not 
interfere at all with her ability to make friends. 

But when we looked at the quantitative data in the peer rating graph, we saw 
almost the exact opposite of what the teacher said. Mary received the lowest score in her 
class, meaning she was the least popular child. 

This apparent disparity can be traced to the very source of the data. Our interview 
data all came from adults, whereas the Peer Rating data came from children. So the 
discrepancy was actually between the perception of adults and that of children. 

This school was located in a low-income, high-crime area where drugs and 
violence were not uncommon. One of the on-going themes of the school was to teach 
children the value of loving and caring of others. When Mary hugged others, the teachers 
perceived her acting out the very value system they were trying to instill among their 
students. As one teacher put it: 

With Mary showing that type of love to them and doing this with them all the time,it teaches them 

that yes, you can be this way. You don’t have to be all hard and everything. You can love somebody. 

It’s OK. And Mary lets them know this. 

The children, on the other hand, did not perceive things the same way as their 
teachers. Rather than feeling loved by Mary, they thought Mary was spitting on them 
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because she had a serious drooling problem. Besides, Mary was tall and strong and did 
not know her own strength. So when she got too close to people, as she often did, she 
could easily hurt her peers. So most of the children did not want to play with Mary, even 
though they could tolerate her. This example shows that what adults thought to be the 
facilitator of inclusion were actually the barriers in the eyes of the children. The 
discrepancy between qualitative and quantitative data lead to a new understanding. 

Family Studies Example 

ECRH is conducting a series of studies with families to obtain their perspectives 
on inclusion. The studies investigate the decision-making process leading to an inclusive 
placement, their perceptions of the child’s program, and participation in inclusive 
activities in the community. This section of the paper focuses on comparing data from 
two separate family studies in one of the 16 ECRH programs: (a) face-to-face, in-depth 
interviews that were part of the ecological systems study, and (b) a telephone survey of a 
larger, representative sample of families using the same program. The interviews were 
conducted to obtain in-depth information on a small sample of families using the 
program, while the survey was used to determine how generalizable our findings from 
the ecological systems study were to other families using the program. 

Program Context 

The context is a large, county-wide school district that offers preschool inclusion 
through community-based placements in child care centers and preschools for 3- to 5- 
year-old children with disabilities. This community-based option was offered for the first 
time during the 1994-95 school year, the year that we collected data for the ecological 
systems study. Typically one or two children with delays or disabilities are placed in a 
regular classroom with typically developing children. 

Special education services are provided to the children through an itinerant 
collaborative model. The early childhood special education (ECSE) teachers and related 
services professionals (speech, physical and occupational therapists) work with the 
classroom early childhood teacher to systematically embed the child’s IEP (Individualized 
Education Program) goals in ongoing classroom activities and routines. Special services 
are usually provided to the child in the classroom. The ECSE teacher and therapists 
typically work directly with the child in the classroom as well as collaborate with the 
classroom teacher and related services personnel on meeting the child’s IEP goals. 

Interview Data 



In-depth, face-to-face interviews were conducted with the parent(s) of five 
children with disabilities and two typically developing children. A semi-structured 
interview guide was used. The protocol asked about the child, the child’s program and 
service history, the decision making process about the program, perceptions of the 
program, as well as the child’s peer relationships and participation in family and 
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community activities. The interviews were audiotaped and transcribed. Families were 
paid $25 for their participation in the interview sessions. 

Survey Data 

The survey questionnaire was administered through a telephone interview 
conducted with the parent or primary caregiver. The survey instrument consisted mainly 
of precoded close-ended questions. The questionnaire is divided into five sections and 
covered this information: (a) demographic information on the child and the child’s 
disability; (b) the child’s current and previous programs; (c) decision-making about the 
child’s program and educational goals, services that the child receives, sources of support 
for the family, and the parent’s perspective on what is important in the child’s program 
and recommendations for program improvement; (d) demographic information on the 
family and community; and (e) the child’s participation in activities outside the school 
program, sources of friendships and parent’s satisfaction with the child’s peer 
relationships, and perceived limitations on the child’s participation in community 
activities. 

The target population for the survey was the f amili es using co mmuni ty-based child 
care centers or preschools. The research office prepared a recruitment letter with a 
consent form that was addressed and mailed by the school system preschool office. When 
the school system received the consent, it released the family’s name and contact 
information to the research office. Research staff scheduled the telephone interview 
appointments with the parent, and sent a copy of the questionniare for the parent to 
preview and follow during the actual interview. Of the 68 eligible f amili es using the 
program at that time, 34 consented to participate in the survey and 28 actually completed 
the telephone interview, resulting in a response rate of 41%. Families were paid $10 for 
their participation. The respondents were representative of the population of f amili es 
using the program on demographic characteristics (e.g., ethnicity, f amil y composition and 
SES, and parent’s education level) and on the range and types of disabilities. 

Data Analysis 

The framework for the analysis and comparison of the famil y interview and survey 
data is presented in Figure 2. The process followed is different from the classroom data 
analysis because that analysis effort involved using and comparing different types of 
measures and data sources (e.g., children, teachers) on the same group of seven children. 
In contrast, the two types of family measures were administered to different fa mili es 
using the same program. The seven f amili es with whom we conducted face-to-face 
interviews were not included in the survey sample. 

[Insert Figure 2] 
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Figure 2. Analytic Framework for Family Qualitative and Quantitative Data 
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Data Reduction 

The members of the site research team independently coded different interview 
transcripts, and prepared an analytic memo on each of the seven children about issues 
related to inclusion. After sharing each summary memo, the research team developed a 
list of eight themes for further analysis. Data were further reduced by preparing a theme 
matrix on each child. In this step, we displayed summary information we had on each 
theme for the child. This type of data display facilitated comparison and aggregation of 
individual child information. 



Analysis of the telephone interview data proceeded through the steps of coding, 
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research methods. The results were presented in a narrative report with a two-page 
Executive Summary, which was disseminated to the participating school system and 
families who requested the survey results. 

Data Comparison and Integration 

We used the constant-comparative method to continue analysis of the qualitative 
data from the seven cases. From the theme matrices, we combined information across 
the cases and discussed working hypotheses for each theme. Negative case examples 
were used to refine, and sometimes to qualify, the interpretations we were making. We 
developed a one-page summary with working hypotheses on each theme. 

At this point, we had summary data for each method in a different form: a two- 
page Executive Summary for the survey data, and the theme summaries for the interview 
data. To compare this information more easily, the summary data were displayed in a 
table format organized by themes. Table 2 presents a sample of summary data on six of 
the eight themes from the interviews, organized by the theme names. 

[Insert Table 2 here] 

Having described the analytic process for these two types of family data, we now 
explain how the purposes of development, expansion, and complementarity played out in 
our analysis. Examples of substantive findings are presented for the sake of illustration of 
methodological issues (additional information on specific findings is available in 
Marquart, 1996). 




w 
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Table 2. Comparison of Information from Interview and Survey Data 



Theme 


Face-to-face Interview 


Telephone Survey 


1. How and why child 
was placed in program 


Two aspects of decision: 

(1) Community-based "inclusive" 
option 

(2) Specific child care center 
Factors affecting choice: 

• Visited and liked classroom & 
teacher 

• Convenience of location 

• Flexibility in hours 

• Good reputation of center 

• Concern if center would accept child 
because of behavior 


Parents’ most important reasons 
for using program: 

• Offers special educ. services 
or therapies 

• Provides opportunities to 
learn 

• Provides opportunities to play 
with other children 


2. Program’s appropri- 
ateness for child 


In successful placement, there’s a 
"match/fit" that occurs between child’s 
and family’s needs & program. Factors 
affecting "match/ fit": 

• Acceptance by staff & children 

• Likes activities and routines for child 

• Child likes program 

• Sees benefits or specific 
improvements 


• 90% said very important for 
child to be in inclusive program 
® 80% indicated child usually/ 
always receives special services 
needed 

• 86% were satisfied with way 
in which child’s educational 
goals were made 


3. Meaning of inclusion 


After inclusive option became 
available, parental shift in expectation 
from "Why should child be placed in 
inclusive option?" to "Why not?" 

• Expectation that child will be with 
typical children because always has 
been 

• Belief that staff has inclusive attitude 
and personal investment in inclusion 

• Importance of role models & 
learning from typical children 

• Benefits to child with disability and 
typical children in class 
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4. Peer relationships 


• Program is primary setting in which 
peer interaction & friendships (other 
than with siblings) occur (program is 
where preschoolers spend significant 
part of day with other children). 

• 5 children have peer relationships 
both outside of & at their program. 2 
children lacked specific social 
interaction skills to develop 
relationships. 

• When disability is a limiting factor 
in developing relationships, it limits 
more in access to Deers rather than the 
quality of interactions with Deers. 


• Families were more satisfied 
with opportunities for child 
friendships at the program than 
outside the program 

• Child plays with other 
children with disabilities slightly 
more outside the program than 
at the program 


6. Helpful and non- 
helpful players 


Characteristics of helpful players: 

• Consistent presence over time & 
settings 

• Personal investment in child 

• Provides different types of support 

• Dependable source of information 
about child 

Characteristics of non-helpful players: 

• Minimize/disregard family concerns 

• Inadequate communication 


The most helpful supports were: 

• Other family members at 
home 

• Child’s teachers 

• Other professionals in 
community & at child’s program 


7. Child’s participation in 
family & community 
activities 


Factors that affect participation: 

• Parent’s safety concerns about child 

• Parent’s perception of what is 
expected of child’s behavior 

• Lack of other young children in 
immediate neighborhood 

• Family’s own style, schedule and 
how it participates in the community 

• An extended family system was so 
strong a part of family’s culture that 
family did not need/choose to 
participate much in the community 

• Young age of children 


Limitations on participation: 

• Child’s language skills 

• Family’s schedule and time 
constraints 

• Attitudes of other towards 
child’s disability 

• Child’s behavior 

• Lack of other children to play 
with in neighborhood 



Development . Previous mixed-method work has used interview and survey 
methods sequentially or for a development purpose (Greene et aL, 1989). We used 
multiple methods in an iterative, sequential process to develop subsequent data 
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collection activities an d to expand interpretation of data collected. That is, we began our 
investigative process with in-depth interviews with a small sample of families. The 
information collected from those interviews helped in the conceptualization of the 
content domain of the survey questionnaire that was administered to the broader sample 
of families. In turn, the information from the questionnaire opened up yet new avenues 
to explore in follow-up interviews with famili es. This circular, iterative process enhanced 
our ability to build on and expand the understandings that we were developing. It 
allowed us to incorporate new learnings and working hypotheses into subsequent data 
collection activities to make more complete interpretations of our data. 

Expansion . Expansion extends the breadth of inquiry by using different methods 
for different inquiry components. Different methods allowed us to ask different kinds of 
questions and get different types of information from families. On surveys it is easier to 
use Likert-type scales to obtain measures of satisfaction, importance, frequency, and so 
on. For example, it was important in our study to obtain a quantifiable measure of how 
impor tan t an inclusive program was to families, the extent to which the child receives 
needed special services, and families’ satisfaction with the decision making process about 
their child’s educational goals. As shown in Table 2 (section 2), high proportions of 
families believed an inclusive program was important for their child and were satisfied 
with the process for deter minin g the child’s educational goals. A slightly lower proportion 
indicated that the child usually or always received the special services needed. 

The open-ended interviews, on the other hand, allowed us to understand systems 
and process interaction variables and underlying reasons for families’ perceptions. For 
example, we could explore the decision making and program placement process for 
individual famili es. This information led us to develop a working hypothesis about the 
"match/fit" that occurs between the child’s and family’s needs and the program in a 
successful placement (see Table 2, sections 1 and 2). When we were able to explain the 
dynamic of this process, the separate pieces of information about what is important to 
famili es in the program obtained from the interviews and the survey began to fit together 
holistically. This holistic understanding is one of the purposes and benefits of qualitative 
methods, but it also enhanced our interpretation of our quantitative data. 

Some types of information can best be captured by one method and are not 
appropriate or feasible through the other. For example, one of the institute research 
questions focuses on participants’ definition or meaning of inclusion. This type of 
question can not be answered in a meaningful way through a written or telephone survey 
item. As shown in Table 2, section 3, we obtained information on participants’ meaning 
of inclusion for their child through the in-depth interviews. The face-to-face format 
allowed us to ask about inclusion in the context of the child’s program and the family’ s 
circ umstan ces, with opportunities for probing and eliciting more explanatory information. 
This is also a highly inferential theme and draws from other information obtained during 
the interviews. (This was also true for the two other themes omitted from the table, 
Communication with direct service providers, and Expectations for the child.) 
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Complementarity . The use of multiple methods allowed us to measure 
overlapping but also different facets of the phenomenon of inclusion, giving us an 
enriched, elaborated understanding of the phenomenon. For example, on the survey, we 
systematically asked families how helpful different informal and formal sources of 
support had been to them (see Table 2, section 6). In our interviews, families referred to 
individuals who had been supportive to them in meeting their children’s needs. It was 
notable who they mentioned (including the bus driver!), as well as who they neglected to 
mention (our small sample never mentioned the related services therapists who worked 
with their children, for example, and from our classroom observations we knew why). But 
the survey respondents considered the related services personnel very helpful. Without 
the survey information, we would have drawn incorrect conclusions about the role of 
related services professionals in the child’s program. 

From the interviews, we also obtained additional information about characteristics 
of helpful and non-helpful players to the family (Table 2, section 6). Helpful players had 
a consistent presence in the family’s life over time and settings, had a personal 
investment in the child, provided different types of support, and were a dependable 
source of information about the child. Non-helpful players were seen as minimizing or 
disregarding family concerns and providing inadequate co mmuni cation. In turn, these 
characteristics contributed to our understanding of why families rated as most helpful 
other family members, the child’s teachers, and other professionals at the program and in 
the community. 

The interview and survey responses about the child’s participation in family and 
co mmuni ty activities also served a complementarity purpose (Table 2, section 7). From 
both methods we obtained similar information about limitations to participation, such as 
lack of other young children in the neighborhood, the family’s schedule and time 
constraints, and the child’s behavior. From the survey, we learned that the most 
frequently reported limitation was the child’s language skills, which did not arise in our 
interviews with the small sample of families. From the interviews, we learned first-hand 
that the family’s own style and way of participating in the community was perhaps the 
most important determinant of the child’s participation, and that an extended family 
could meet the family’s needs so well that it did not need or chose more extensive 
participation in the community. That type of in-depth information could only be gained 
through more extensive contact with families. 

Further Data Integration 



The next steps in data integration will proceed on several levels. First of all, the 
survey results from this program will be aggregated with survey information from the 
other programs within our site and across the institute sites. Similarly, the eight themes 
initially developed will be "tested" by continuing to apply the constant-comparative 
method across other programs within site and across site. This qualitative analysis needs 
to be done carefully so as to preserve the meaning of the theme in the program context, 



but still to be able to undercover common themes across families. 



These initial themes 
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will surely be refined and perhaps be expanded as analysis continues across programs. At 
that stage, we will need to determine which types of further integration are meaningful 
and feasible. 

Conclusion 

This paper has dealt with how qualitative and quantitative methods were mixed 
for data collection, analysis and interpretation within classroom and family studies that 
are being conducted to investigate the phenomenon of inclusion from multiple 
perspectives. The data from each method were used to truly inform the other, resulting 
in an understanding of inclusion that is "more than the sum of the parts." 

The analytic steps of data reduction, comparison, and integration represent an 
ongoing, iterative process with mixed methods. As the research team proceeds with 
integration of data across programs, we will need to recycle back and do further types of 
data reduction and comparison. With the larger number of cases that will be aggregated, 
we will also transform some of the qualitative information into numbers to determine the 
commonalities across cases while preserving the discrepant information from "negative" 
case examples and their context. 

The Greene et al. (1989) framework provides a useful framework for 
conceptualizing research designs that employ multiple methods. However, we found that 
the five purposes defined by Greene et al. do not necessarily represent discrete, mutually 
exclusive categories. For example, we found that it is difficult to distinguish aspects of 
the initiation and expansion purposes. Also, in mixed methods studies as complex as the 
ones we are conducting, we found that issues related to purpose, design, and analysis are 
often confounded when attempting to apply the five categories. That is, these categories 
may take on different meaning when applied to purpose, design, and analysis issues in a 
mixed methods study. In fact, we may have confounded them in our discussion of the 
classroom and family examples. We believe that in future work it would be useful to test 
out this notion by applying the separate categories to purpose, design and analysis 
components of mixed method studies. 
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